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Abstract 

This study is an investigation of how modeling aids learners in developing 
problem solving skills within a computer learning environment. The task given 
to 45 subjects was to solve a crime in the computer game. Where In Time Is 
Carmen San Diego . Three subject groups received one of the following 
treatments: Expert Model, Novice Model, or No Model. Modeling entailed 
subjects viewing a computer-based animated and narrated movie showing how 
either an expert or novice completed the task. Results of the study indicate that 
modeling does not have an effect on the performance of participants, however it 
did contribute to the development of cognitive skills. 
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The Effects of Modeling to Aid Problem Solving 
in Computer-Based Learning Environments 

The capabilities offered by computer-based learning (CBL) systems have led 
to significant innovations in instructional delivery systems, specifically in the 
design of simulations, demonstrations and drill-and-practice. The design of 
many CBL programs is linear and i^ystematic. However, new paradigms for 
implementing CBL, such as hypermedia, require nev/ design models and 
prescriptions to guide the development of instructional systems. 

This study focuses on the instructional method of modeling and its 
application in a CBL environment. It is imperative that the importance of 
modeling is not misunderstood. We believe that there is a need for a more 
effective way to facilitate cognitive learning in a CBL environment, and that 
modeling delivered by computer is a strateg)-' v/ith considerable potential. 
Specifically, we want to determine how computer-based modeling of strategies 
within a CBL environment influences the acquisition of cognitive skills. 

Theoretical Background 

The history of modeling includes such theorists as Miller & Dollard, Bandura, 
and even Aristotle. Their work focused mainly on the concepts of imitation and 
observational learning. Aristotle noted that an individual, "Is the most imitative 
of living creatures, and through imitation learns his earliest lessons; and no less 
universal is the pleasure felt in things imitated." (Butcher, 1922, p. 15) More 
recently, the social learning theorists Miller and Dollard (1941) offer a cogent 
description of the role of imitation and social learning, emphasizing that 
imitative learning is an acquired social behavior. Bandura (1971) conducted 
extensive research on psychological modeling in the behaviorist realm. 
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A comparison of definitions of modeling is useful for isolating attributes of 
this instructional treatment Bandura (1971) describes modeling using a broad 
range of terms, "Among the diverse terms applied to matching behavior are 
'imitation', 'modeling', 'observational learning', 'identification', 'internalization', 
'introjection', 'incorporation', 'copying', 'social facilitation', 'contagion', 'and 'role 
taking'" (p. 3). Romiszowski (1981), on the other hand, gives an apt description 
of modeling as the shov^ing, not telling, a learner some set of processes, steps, 
skills and strategies to solve a task. Davies (1981) and Gagne (1987) define 
modeling simply as "demonstrations." 

More recently, Welsh (1992) differentiates two types of modeling, behavioral 
modeling and cognitive modeling. We've attributed behavioral modeling to 
much of the research we've illustrated in the preceding paragraphs. Cognitive 
modeling, on the other hand, incorporates systematic verbalization on the part of 
the model to make his or her thinking processes visible to others. Thinking aloud 
and mental modeling are two strategies that describe what occurs during 
cognitive modeling. 

Behavioral Modeling Research 

"Algorithms guarantee solutions: if properly carried out they always yield the 
correct answer." (Perkins, 1986:198). Demonstrative modeling is, in essence, an 
algorithm, since it is the objective of the modeler to show the learner the perfect 
solution to the problem. The research has consistently shown that demonstrative 
modeling strategies for completing tasks have yielded greater performance than 
no modeling (Yaeshima, Ninomiya, Ohnogi & Oda, 1982; LaNunziata, Cooper & 
Hill, 1985; Rivera & Smith, 1987; Baggett, 1987; Smith & Hailey, 1988; Robert & 
Chaperon, 1989). 
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Smith &L Hailey (1988) found that demonstrative modeling techniques 
resulted in better performance in learning hv^asi self-examination over narrative 
explanation and written instructions. In their exp.:riment, the experimenters 
assigned 60 college women 4) one of three instructional treatments: actual breast 
demonstration, synthetic breast demonstration, and no demonstration. After one 
month, the two subject groups who received the demonstration treatments 
performed better than the control group. Yet after three months, all differences 
between experiment and control subjects disappeared. 

Baggett (1987) investigated the use of a narrated film and a correctly 
assembled object as stimulus in the task of assembling a toy helicopter from a 
number of given pieces. The modeling by film consisted of a 15 minute, narrated 
8mm film showing an expert naming and putting together the pieces to assemble 
the heUcopter correctly. The assembled object was a completely assembled toy 
helicopter. Baggett found that any combination of film and the assembled 
helicopter resulted in greater task completion than either treatment in isolation. 

Robert and Chaperon (1988) conducted a linking experiment between 
behavioral and cognitive modeling. In their experiment, they used a Piagetian 
horizontal representation water-level task with college women. Experimenters 
randomly assigned women to one of three groups: cognitive modeling, 
exemplary modeling and no modeling. Cogiutive modeling entailed watching a 
videotape of a woman as she correctly solved three water-level tasks. During her 
solution, she both demonstrated and verbalized her actions. Exemplary 
modeling used the same videotape of the woman performing the task, however 
she never voiced any part of her solution. The no modeling group received no 
modeling of the task. Experimenters immediately evaluated subjects after 
treatment on their ability to solve a similar task with tn.e same water bottles and a 
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task with different water bottles. Fifteen d.' vs ]'ater, experimenters conducted a 
delayed post-test to measure stability of the Aearned behavior. 

The results from this experiment clearly ■..riowed the impact of both cognitive 
modeling and exemplary modeling in enabling women to master the water-level 
task. However, the cognitive modeling compor ent appeared to be the aritical 
component that fostered improvement in th\ i'earner. 

Cognitive Modeling Research 

There is still the question we raised earlier o/ the technique one uses to model 
cognitive processes— i.e., think-aloud or menta^ modeling. Schoenfeld (1985) uses 
think-aloud routines to model the cognitive piocess of solving math problems. 
He has an ongoing contest whereby students pr .sent math problems for him to 
solve. In solving the problem, he thinks out loud, articulating his strategies for 
solving the problem. In essence, he is showing' \he students how he thinks in 
seeking a solution to a math problem, exposing !hem to all the complexities, 
mistakes and pitfalls one must deal with along : he way. 

In teaching reading skills to biology studente. Mikulecky, Clark and Adams 
(1989) evaluated modeling as a means of coaching college freshmen to read texts 
more strategically. In particularly, they used cc .nputer-based presentations of 
text and analysis to model strategies such as ' ie atifying key concepts, writing 
linking summary statements to compare and cc itrast key concepts, and 
graphically mapping relationships among ke " concepts. They found that 
modeling these reading skills for biology stuc ;nts with a CBL program yielded 
higher exam scores over a control group that Jid not receive the CBL lessons. 

Mandinach (1987) compared discovery learrdng and participant modeling as 
a method to develop strategic planning knowledge. Two groups of seventh and 
eighth grade students played a game called Wumpus, The experimenters used a 
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discovery learning paradigm for one group, while the other group received 
participant modeling that systematically modeled appropriate cognitive skills 
and strategies. Students in the participant modeling group performed better in 
task completion; however the discovery learning group performed better in the 
transfer of skills to other tasks. 

Synthesis 

Between both behavioral modeling and cognitive modeling, the key 
instructional elements that comprise models are demonstration and 
verbalization. While demonstration is an established construct, verbalization is 
somewhat more undefined. For example, in Robert and Chaperon's (1988) 
research, verbalization was operationalized by describing one's actions during a 
perfectly demonstrated task. On the other hand, Schoenfeld (1985) also 
verbalizes his solution to math problems,, however this verbalization includes the 
mistakes he is making and how he resolves those mistakes. We believe the 
demonstration of mistakes and verbalization of how to recover from them is a 
more effective instructional strategy for faciUtating the development of cognitive 
skills than modeling perfect solutions. Thus, our research will investigate the 
differences between these two verbalization strategies in the context of cognitive 
modeling witliin CBL. 

Hypotheses 

On the basis of the literature, we expect our study to replicate the findings of 
prior modeling research. 

HI: Participants who are shown a model of how to perform a task will 
perform better on that task than participants who are not shown a 
model. 

• 8 
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How we define the protocol for modeling of cognitive processes is important 
to developing our study. Our primary concern is how experts think about 
solving tasks, and how to teach those thinking skills to others. 

^ In the context of prior modeling research, weVe developed two constructs for 
modeling. The first construct is the expert demonstration model, whereby an 
expert demonstrates and explains how to perform a task perfectly. The second 
construct operationalizes one of the salient features of Schoenfeld's research, 
namely the modeling of how to deal with errors. In this construct, a novice 
performs a task while an expert provides commentary on the novice's 
performance. 

The showing of incorrect solutions is an important distinction to make 
between expert and novice modeling. Expert modeling, as described in the 
literature, tends to show the correct performance of a task, whereas novice 
modeling shows both correct and incorrect (with recovery) solutions. People 
make mistakes every day, and learning how to recover from those mistakes is an 
important skill. We believe the observation of a novice's recovery from incorrect 
solutions contributes greatly to acquiring cognitive skills. Therefore, we predict 
that novice modeling will result in successful completion of a task over the expert 
model since the participant will learn how to avoid and recover from common 
mistakes novices tend to make. 

H2: Participants who observe a novice model while listening to an expert 
comment on the novice's performance will perform better in a task 
than participants observing an expert model. 

Performance is not the only measure of success. Our interest in the cognitive 
skills a learner acquires as a result of instruction drives our belief that modeling 
will have a positive effect on the key strategies a participant uses in solving a 
task. 
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H3: Participants who observe a model will use key task strategies more 
effectively than participants who do not observe a model. 

H4: Participants who observe the novice model will use key task 

strategies more effectively than participants who observe the expert 
model. 

Another measure of a learner's cognitive skill is articulation. Collins^ Brown 
& Newman (1988) suggest that having a learner articulate what they know, by 
explaining how they plan to solve a problem or why they applied a particular 
solution, one can get a sense of the cognitive skills they've learned. Thus, we will 
investigate how the different modeling treatments affect the participant's 
articulat'Ons. 

H5: Participants who observe a model will have stronger articulations 
about how to solve a task than participants who do not observe a 
model. 

H6: Participants who observe the novice model will have stronger 

articulations about how to solve a task than participants who observe 
the expert model 

H7: Participants who have strong articulations will use key task strategies 
more effectively than participants who have weak articulations. 

H8: Participants with strong articulations about how to solve a task will 
perform better in a task than participants with weak articulations. 

Method 

Design 

The experiment was a l-by-3 design. There was one variable, modeling, with 
three levels. The levels were expert, novice, and control. The expert level 
consisted of a modeling presentation showing an expert playing Where In Time 
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Is Carmen San Dieg o (WITICSD) with the expert's vocal narration. The novice 
level consisted of a modeling presentation showing a novice playing WITICSD 
with an expert's vocal analysis of the novice's performance. The control level 
consisted of no modeling presentation. 

Participants 

Fifty-two undergraduate and continuing education students from an 
introductoiy computer course at a large midwestern university signed up to 
participate in the study. The course instructor awarded extra credit to the 
participants for participation. We screened participants for prior experience with 
WITICSD. None of the participants had ever played WITICSD or any of the 
other Carmen San Diego games. Seven participants failed to shew up for their 
appointment, resulting in 45 participants who contributed data to the study. 

Materials 

Materials for the study included a Macintosh® SE computer, WITICSD 
software and documentation, Microsoft Word™ software, custom-designed 
modeling presentation software, written instructions, a pad of paper, and a pen. 

WITICSD is an educational software gurie. In the game, the player takes on 
the role of a detective solving a crime. Solving the crime entails interacting with 
the software to obtain clues, tracking the criminal through time and space, 
acquiring an arrest warrant, and arresting the criminal Players decipher clues 
about the identity of the criminal and the destination he or she is traveling to by 
using the New American Desktop Encyclopedia, which comes with the software. 
We also provided the standard documentation for WITICSD, which includes the 
encyclopedia, instructions on how to use the software and hints for playing the 
game. 

11 
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In order for participants to write their articulations, we used Microsoft Word 
software for word processing. Thus, participants wrote their articulations using 
the computer. 

The custom-designed modeling presentation software consisted of 
MediaTracks™-produced "movies." MediaTracks enables one to record actions 
seen on the computer screen. After one records the movie, one can edit and add 
sound to the movie. The result is an animated, narrated presentation. We 
produced one movie for presenting the study's instructions to the participant, 
one movie for the expert treatment, and one movie for the novice treatment. 

The instruction movie was 3:17 (minutes:seconds) in length. The content of 
the movie entailed screens of text that illustrated the narration of instructions for 
the study (see Appendix A). We also provided a print version of the instructions 
to the participants. 

The expert movie was 7:16 in length. The content of the movie entailed an 
expert playing WITICSD, showing the perfect solution for playing the game. The 
expert's narration consisted of noting tl\e strategies and heuristics he used while 
playing the game (see Appendix B). 

The novice movie was 10:54 in length. On the basis of observations of novices 
playing WITICSD, we identified the most common mistakes novices make. The 
content of the movie was a novice playing WITICSD. An expert narrated as the 
novice performed. The narrator noted the successful and unsuccessful strategies 
the novice was using (see Appendix C). 

It is important that we make clear the distinction between the expert and 
novice movies. The expert movie emphasized a perfect path solution to the task 
and illustrated advanced techniques for solving the problem whereas the novice 
movie illuscrated errors and theix outcomes and solutions in the course of 
playing the game. 



Modeling in CBL 
12 

Additional materials for the study included a blank pad of legal paper and a 
pen. 

Procedure 

Pre-Treatment Activities , Participants signed up for individual, one hour 
time slots over a one month period in the late fall. To confirm their 
appointments, we contacted each participant by phone or E-mail. We told 
participants that they would be user-testing some educational software. 

We conducted the study in the conference room of a university building. The 
computer was on the conference room table. The blank legal pad and the pen 
were to the left of the computer, and the instructions, program documentation 
and encyclopedia were to the right of the computer. The experimenter sat to the 
back and right of the participant in ordei to observe all of the participant's 
actions. 

When the participant arrived at the testing site, the experimenter explained 
the nature of the experiment to the participant as well as issues of confidentiaUty 
and voluntary participation. In order to decrease anxiety, the performance of the 
software, not the participant, was emphasized. The experimenter told 
participants that they would be testing some educational software and that our 
interest was in the software's performance, not theirs. 

Next, the experimenter pointed out the resources the participant had at his or 
her disposal throughout the study. The experimenter showed the participant the 
documentation, instructions, encyclopedia, notepad and pen. 

The experimenter then played the instruction movie for the participant. Prior 
to playing the movie, the experimenter reirunded participants that they had a 
printed copy of the written instructions that they could use at any time during 
the experiment or to follow along during the movie. 
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Since the focus of this study was on problem solving strategies and not 
procedural knowledge of using the software and hardware, we wanted to make 
sure all participants were comfortable in the procedural aspects of the Macintosh 
and WITICSD before receiving treatment. After the participant had viewed the 
instruction movie, the experimenter would explain the basic use of the computer 
and the mouse pointing device. This required that the participant demonstrated 
proficiency in using the mouse to interact with the computer. We always 
observed accomplishment of this exercise. 

In order to make the participant familiar with WITICSD user interface, the 
experimenter conducted a practice task. In this task, tlie experimenter guided the 
participant in using tlie WITICSD software by exploring one location. The 
experimenter explained how to click the various buttons, obtain clues, and travel 
from one location to another while the participant clicked the appropriate 
buttons. The experimenter told the participant that this was a practice task for 
their familiarity only and he encouraged them not to be concerned about solving 
the task successfully. Participants could ask questions about the functional use of 
the software and hardware only during this practice task. 

These orienting activities and associated practice not only helped the 
participants learn about how to use the software, but helped form the context for 
the task itself, especially the modeling. 

Treatments . The experimenter randomly assigned participants to one of the 
threes treatments, expert, novice and control. For the expert and novice 
treatments, the experimenter prefaced the showing of the appropriate movie 
with the statement that the participant will now see a short movie that will give 
tlie participant more information about playing WITICSD. The experimenter 
encouraged the participant to do nothing else but watch the movie. The control 
treatment group did not watch any movie. 

' 14 
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Articulation Task > Upon completion of the treatment, the experimenter 
opened the word processing software and instructed the participant to answer 
the question, "Explain to a friend how to play Where In Time Is Carmen San 
Diego." Each participant had five minutes to answer the question. When the 
participant finished his or her answer, the experimenter saved the answer in a 
file named after the participant. 

Game Task . At this point, the participant played a single game of WITICSD. 
The experimenter set up the software so that all participants played the same 
game, the stolen Sultan's Howdah, In order to avoid participants who, lacking 
motivation to work efficiently, simply randomly pressed buttons to finish the 
experiment quickly, we offered a $10.00 prize to the participant who completed 
the game with the most remaining chronotime (a measure of efficiency used in 
the game). 

Dependent Measures . During the task, the experimenter recorded data using 
a data tabulation instrument (See Appendix D). The experimenter recorded data 
for each location the participant visited. 

• Location/Date . The location and date the participant navigated to 
while playing WITICSD. For example, France, 1892. 

• WITICSD Chronotime . The chronotime displayed on the game screen 
was recorded when the participant first entered a new location. When 
the participant made a successful arrest, the ending chronotime was 
also recorded. Unsuccessful arrests resulted in a chronotime of 0 being 
recorded. 

• Actual chronological time . Time was recorded to the second at the 
beginning and conclusion at each location and for the entire game. 

• Witness . One point (or hash mark) was recorded each time the 
participant went to the witness for dues. If the participant had the 
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option to obtain more information while viewing the witness, an 
additional mark of "m" indicated that the participant viewed that 
"more" information, and a mark of "o" indicated that the participant 
did not view more information. If no "more" option was available, a 
dash (-) was recorded beside the hash mark in the witness box. 
Informant . One point was recorded each time the participant went to 
the informant for clues. If the participant had the option to obtain 
more information while viewing the informant, an additional mark of 
"m" indicated that the participant viewed more information, and a 
mark of "o" indicated that the participant did not view more 
information. If no "more" option was available, a dash (-) was 
recorded beside the hash mark in the informant box. 
Scanner . One point was recorded each time the participant went to the 
scanner for clues. An option to scan further was come across 
randomly after an initial scan was completed. An "f" was recorded 
beside the scan hash mark if the scan further option was taken. 
Note Taking . One point was recorded each time the participant wrote 
a note. For example, if the participant wrote a aote, the performed 
another recordable activity, such as using the encyclopedia, then wrote 
another note, two points would be recorded. On the other hand, if the 
participant wrote a note, thought for a while, then wrote another note, 
only one point would be recorded since the participant did not 
perform a recordable activity between note writing. 
Reference . One point was recorded each time the participant used the 
New American Desktop Encyclopedia to look up information. For 
example, if the participant used the encyclopedia, then performed 
another recordable activity, such as taking a note, then referred back to 

16 



Modeling in CBL 
16 

the enq^clopedia, two points would be recorded for this measure. On 
the other hand, if the participant used the encyclopedia, thought for a 
while, then used the encyclopedia again, only one point would be 
recorded since the participant did not perform a recordable activity 
between the two instances of using the encyclopedia. 
Program Documentation . One point was recorded each time the 
participant used the program documentation to look up information. 
For example, if the participant used the program documentation, then 
performed another recordable activity, such as taking a note, then 
referred back to the program doomientation, two points would be 
recordedfor this measure. On the other hand, if the participant used 
the program doomientation, thought for a while, then used the 
program documentation again, only one point would be recorded since 
the participant did not perform a recordable activity between the two 
instances of using the program dooomentation. 
Select Clues . One point was recorded for each plain clue, such as sex, 
eye or hair color, tlie participant entered into the crime computer 
database. 

Subtle Clues . One point was recorded for each subtle clue, such as the 
criminal's favorite author or artist, the participant entered into the 
crime computer database. Randomly, the game would present hair or 
eye colors which required some inferential problem solving skill. For 
example, "she had hair the color of mahogany" is an inferred hair color 
clue. When such a clue was correctly recorded in the cri le computer 
database, one point was recorded in the inferred clue box. 
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• Arrest Warrant Attempt . One point was recorded each time the 
participant attempted to get an arrest warrant by clicking the Compute 
button in the crime computer database. 

• Arrest Warrant One point was recorded when the participant 
obtained an arrest warrant. 

• Task Complete . One point was recorded when the participant 
completed the task by arresting the suspected thief. This means that if 
the participant falsely arrested a thief, the task was recorded as. 
complete even though it was not successfully completed. 

• False Arrest . One point was recorded if the participant falsely arrested 
a suspect, by either not having an arrest warrant or having an arrest 
warrant for the wrong criminal. 

• False Leads . One point was recorded if the participant followed the 
suspect to an incorrect destination. 

Success in completing the task, which we call outcome, was operationalized 
in the following manner. We scored participants who acquired the correct arrest 
warrant, successfully tracked the thief, and made the arrest as 3. Participants 
who acquired the correct arrest warrant or tracked the thief and made the arrest 
were 2. Participants who neither obtained the correct arrest warrant nor tracked 
the thief and made the arrest were 1. 

We determined the effectiveness in completing the task by scoring and 
combining the four key task strategies, witness, informant, scarmer, and notes, 
into one measure, called effectiveness. We judged a participant to be effective in 
their use of key strategies if they don't use the witness and informant more than 
once for each location, don't use the scanner at all, and take at least one note at 
each location. 
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Scoring each of these strategies was in the following manner. If the 
participant accessed the witness clue one or fewer times per location, we 
awarded one point. Accessing the witness clue two or more times resulted in no 
points. We totaled the points, then divided by the total number of locations 
visited by the participant, resulting in a score between 0 and 1. We used the 
same methodology to calculate the score for the informant. Scoring for the 
scanner was similar to the witness and informant, however not using the scanner 
resulted in one point per location, while using the scanner at a location resulted 
in no points. Scoring notes was also similar to the witness and scanner, with one 
point awarded per location if there was at least one note, and no points awarded 
if there were no notes. We calculated a mean score for the effectiveness of the 
key strategies by adding the scores for witness, informant, scanner, and notes, 
then dividing by four. 

Two raters judged the articulations. They classified the articulations into four 
equal sets of bad, good, better and best articulations. We then combined these 
articulations into two equal sets, with the bad and good sets classified as weak 
articulations, and the better and best sets as strong articulations This rating 
resulted in an inter-rater reliability of .75. Differences were discussed and 
resolved by the two raters. 

Debrief . At the conclusion of the experiment, the experimenter explained to 
the participant the actual purpose of the study. In addition, the experimenter 
was able to ask the participant several informal qualitative questions about his or 
her performance. We wrote the responses to these questions on the participant's 
data sheet. Participants were also able to ask the experimenter questions about 
the study. We concluded the study by thanking the participant for their help and 
assuring them they would receive extra credit for their class. 
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Results and Discussion 
Hypotheses One and Two 

Our first hypothesis predicted that both the novice and expert modeling 
would result in better task performance than the control. We determined task 
performance through two measures, outcome and chronotime. Participants who 
obtained an arrest warrant and tracked the thief successfully received credit the 
remaining chronotime WITICSD. This number ranged from 1 to 21, whereby the 
higher the number, the more successful we judged the participant. All other 
participants received 0 chronotime illustrating their failure in successfully 
completing the game. 

We used a one-factor ANOVA to analyze these variables. There was no 
significant difference between the modeling and control treatments for both the 
outcome and chronotime variables. Thus, there was no support for hypothesis 
one. This finding contradicts the findings of many of the modeling studies 
referenced in this paper, whereby modeling increased a participant's task 
performance. 

There are a number of possible explanations for why we did not obseive 
significant differences between the expert, novice and control treatments. First, 
the expert and novice modeling may have contributed to a false sense of 
expertise among participants. That is, by watching the modeling movie, 
participants became overconfident in their ability to play the g^me. To further 
support this explanation, data from the debriefing shows that six participants 
mentioned that the modeling movie contributed to them ignoring the program 
documentation. As one participant said, "I forgot about the instructions after 
watching the movie." 

A second explanation suggests an opposite phenomena for the control 
Instead of having a false sense of expertise and being overconfident in their 
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ability to solve the task, the control group struggled with the problem. Thus, 
they may have had to use a greater set of problem solving heuristics and learn 
more from their trial and error. Although the finding was not significant, the 
control took longer (M=1121 seconds) to complete the task than either the expert 
(M=795 seconds) or the novice (M=o54 seconds). This may indicate that the 
conlrol was thinking more about tlie task than either the expert or novice. 

A third explanation is the nature of the task, a computer game. While none of 
the participants had ever played WITICSD, a number of participants indicated 
that they had played other computer or video games before. Thus, the successful 
performance of the control group may be due to the skills and strategies 
participants developed while playing other computer games. 

A fourth explanation is orienting activities. Perhaps the orientation and 
instructions given to each participant revealed too much game strategy, 
especially for the control group. 

We found no support for our second hypothesis either. Our second 
hypothesis predicted that novice modeling would result in better task 
performance than expert modeling. For this analysis, we used the same outcome 
and chronotime dependent variables as in hypothesis one. A one-factor ANOVA 
analyzed the variables. There was no significant difference between novice and 
expert modeling. 

The lack of difference between the novice and expert modeling may be 
caused by the novice group not observing successful task completion. We 
anticipated that the novice modeling would result in stronger performance since 
the participants would learn to avoid the common mistakes novices tend to 
make. However, the novice in the novice modeling movie did not successfully 
complete the task, whereas the expert in the expert modeling movie did. Thus, 
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participants in the novice group may not have had a clear idea of how to 
recognize successful task performance. 
Hypotheses Three and Four 

Hypothesis three predicted that expert and novice modeling would result in 
better use of key task strategies than the control, and hypothesis four predicted 
novice modeling would result better use of key task strategies than expert 
modeling. As defined above, we created a dependent measure called 
effectiveness by scoring the participant's use of witness, informant, scanner, and 
note taking. Thus, we compared expert and novice modeling to the control by 
examining the effectiveness. A one-factor ANOVA analyzed the variables. 

A significant difference (Df 2:42, F=6.32, p<.01) was found between the expert 
(M=.809), novice (M=.841) and control (M=.693) treatments. Additionally, a 
Fisher PLSD test revealed that novice was significandy different from control 
(.088, p<.05) and that expert was significantly different from, control (.091, p<.05). 
Thus, hypothesis three has support. While hypothesis four appears supported, 
with novice modeling having a slightly higher mean than the expert modeling, a 
follow-up Fisher PLSD test between expert and novice yielded no significance. 

Although we found no significance with hypothesis one and two, the 
significance found in hypothesis three identifies a key effect of the expert and 
novice modeling. From the results, we can infer that participants who viewed a 
model learned to use the key task strategies better than the control. Additionally, 
there is slight evidence that the novice group learned to use the key task 
strategies better than the expert group. 

One would expect that if participants used the key task strategies effectively, 
they would successfully complete the game. However, the data show an 
incongruence between hypothesis one and three, and hypothesis two and four. It 
is possible to explain this incongruence because while participants in the expert 
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and novice modeling treatment groups may have used key strategies more 
effectively than the control group, we have no direct way of knowing how they 
interpreted the information gained from exercising those strategies. 
Interpretation can only be inferred by a participant's successful completion of the 
game. If a participant successfully completed the game, then we can infer that 
they were successful in interpreting the clues. The key task strategies are only a 
means of uncovering clues necessary for successful completion. Thus, they have 
only a partial role in predicting successful completion. 
Hypotheses Six through Eight 

Our fifth hypothesis predicted that novice and expert modeling would result 
in stronger articulations than the control. We used a one-factor ANOVA to 
analyze the variables. We found no significant difference, indicating that neither 
novice nor expert modeling restdted in stronger articulations by the participant. 
Hypothesis five had no support. 

Hypothesis six predicted that novice modeling would result in stronger 
articulations than expert modeling. A one-factor ANOVA analyzed the 
variables. There was no significant difference, thus the novice model did not 
cause stronger articulations than the expert model. We did not support 
hypothesis six. 

Our seventh hypothesis was similar to the third hypothesis except we 
predicted that participants with strong articulations would use key task 
strategies more effectively than those with weak articulations. Independent 
raters scored the quality of articulations as either good or bad. Thus, we 
compared the good and bad articulations with effectiveness using a one-factor 
ANOVA. There was no significant difference, indicating that the quality of 
articulations did not affect the use of key strategies. Hypothesis seven had no 
support. 
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The explanation of why there was no difference for this hypothesis is similar 
to the explanations presented for hypotheses one and two. It is possible that the 
novice and expert modeling caused a false sense of expertise in participants, 
resulting in weak articulations for some. Additionally, the orientation activities 
may have provided too much information about how to play the game. 

Hypothesis eight predicted that strong articulations would result in better 
task performance than weak articulations. For this analysis, we used the 
outcome and chronotime dependent variables used in hypothesis one and two as 
a measure of task performance. We also used the same articulation ratings found 
in hypothesis six and seven. A one-factor ANOVA analyzed the variables. 

The results of this analysis were significant for both outcome (Df 1:43, 
F=4.648, p<.01) and chronotime (Df 1:43, F=10.078, p<.01). A Fisher PLSD teyt 
(.449, p<.05) revealed that participants with articulations classified as "good" had 
a mean outcome score of 2.435 while "bad" articulations had a mean outcome 
score of 1727, A Fisher PLSD test for chronotime (3.908, p<.05) showed that the 
mean chronotime for "good" articulation was 6,087 while mean chronotime for 
"bad" articulation was 1.909. On the basis of these results, there was support for 
hypothesis eight - strong articulations contribute to successful task performance. 

This finding shows the importance of designing instructional interventions 
that contribute to forming strong articulations, since strong articulations are a 
good predictor of success. We should consider designing instructional 
interventions with the goal of enabling the learner to form strong articulations. 
This strategy contradicts the traditional concern for the terminal goal — 
successful performance on a task. Facilitating successful performance was our 
paradigm v^hen we designed the expert and novice modeling movies. An 
articulation-focused design strategy changes our paradigm to where we are more 
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concerned with enabling learners to talk about how they solve problems rather 
tihan strictly focusing on enabling them to complete the task successfully. 

Summary and Conclusion 

The results of this study offer a number of interesting findings. We were 
unable to repUcate the results found in previous modeling studies in which 
modeling enabled participants to perform better on a task. We also found no 
difference between the novice and expert groups in terms of task performance. 
However, we did find that novice and expert groups used key task strategies 
more effectively than the control group, and that there may have been only a 
slight difference between the novice and expert groups. 

Our articulation task yielded similar results. We found no differences for llie 
expert, novice and control treatments on the quality of the articulations. 
Additionally, participants with good articulations did not use key task strategies 
any differently from participants with bad articulations. However, participants 
with good articulations performed better on the task than participants with bad 
articulations. 

Why didn't our findings support our many of our key hypotheses? We 
beUeve the main reason is the effect of the orienting activities. In order to make 
sure participants were technically able to use WITICSD, we conducted an 
extensive set of orienting activities common to all treatment groups. While we 
accomplished our goal of enabling participants to be technically competent, we 
may have told them too much. This might account for the lack of difference in 
task performance and articulation quality between all treatment groups. 

There are two key implications from our stu.dy. First, cognitive modeling, 
regardless of the verbalization technique used, aids learners in adopting key task 
strategies. Thus, one sho^ild adopt cognitive modeling as an instructional 



Modeling in CBL 
25 

strategy if the instructional objectives require the learning of heuristics. Second, 
since the quality of articulation is a predictor of task performance, we should 
design instruction to achieve quality articulations. Learners who can explain 
how they will solve a problem will experience more successful task performance. 

Future research in the area of cognitive modeling should continue to explore 
variations in verbalization and their effects. Researchers should take time to 
consider the orienting activities, especially when working in a computer-based 
training environment. Additional research should discover how to facilitate 
quality articulations in instructional programs. 
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Appendix A - Instructions Transcript 

You will be playing a computer game called Where In Time Is Carmen San Diego. The object 
of the game is to catch a thief who has stolen a historical treasure. 

In order to catch the thief, you must do three things: 

1) You must gather clues to determine tlie thief s identify and to follow the thief s trail. 
Clues are gathered by using the search button, followed by either the Witness, Informant or 
Scanner buttons. The Witness and Informant buttons may contain dues about the thief s identity 
and his or her next destination. The Scanner button contains clues only about the thief s next 
destination. 

2) You must follow the thief s trail from country to country. The Travel button enables you 
to travel to the next destination. Use the dues you've gathered to determine the next destination. 

3) You must obtain an arrest warrant before arresting the thief. Arrest warrants can b^ 
obtained through the Data, then Evidence buttons. Record the clues youVe gathered and use the 
Compute button to determine if the clues match a criminal and enable you to obtain an arrest 
warrant. 

Here are some hints for playing the game: 

1) It's a good idea to keep notes on paper as you gather dues. 

2) Remember to work quickly and efficiently - you only have a certain amount of time to 
complete the task. Time is subtracted only when you a) travel from one location to another (here 
the time used is approximately 3-5 hours), b) obtain clues from the witness, informant or crime 
scene scan (here the time used is approximately 1 to 2 hours), or c) compute the evidence to try 
and obtain an arrest warrant (here the time used is 1-2 hours). No time is counted against you 
when you look up information from the reference sources. 

3) You'll know when you have tracked the thief successfully when you see suspidous 
persons. The suspect you're tracking uses other gang members to check you out when your are 
getting close. If you interrogate a witness, informant or crime scene and they know nothing, 
you're probably in the wrong location. You'll then have to backtrack to the previous location to 
get back on the trail of the criminal. 

4) When the criminal is at the same location your are, he or she launches a sneak attack 
against you. Make sure you have your warrant when this happens or else the criminal will get 
away. If you have your warrant, look to other sources of dues to find the criminal. 

5) The key to tracking the criminal is dedphering the clues you find. You can dedpher 
clues with the New American Desk Encyclopedia. Be sure to look at the "How To Use" section in 
the Encydopedia. 

Here's an example of how to acquire clues and evidence: 

For example, let's say you use the Witness button and the clue is, "She wanted to talk with 
Dag Hammarskjold about economy." You could then look up "Harnmarskjold" in the 
encyclopedia and find out he was a Swedish stateman in the 1900's. This clue would tell you the 
criminal's next destination is probably Sweden in the 1900's. 

In addition to this clue, there is another button called More. You can click this button to get 
evidence about the crinunals identity. For example, if the clue is "She just finished reading the 
book 'Animal Farm'.", you would go to the evidence selector by dicking the data button. You 
would then select each author, and look up in the encydopedia if that author wrote ' Animcd 
Farm'. So in this case, we would find George Orwell and keep him selected as a piece of 
evidence. 
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Appendix B - Transcription of Expert Movie 

This movie is a demonstration on how to play Where in Time is Carmen San Diego. My name 
is Peter and I'll be showing you how to play the game. In this case the Indian Constitution has 
been stolen from India in 1947 by an unknown male. I have written down these facts in my 
notebook. The Federal Time Commission has given us 33 hours to solve the crime. We'll begin 
our investigation by searching for clues at the scene of the crime. I'll first investigate the witness. 
Here I find that the thief wanted to join a group called the Samurai. This gives me some 
information as to the next destination of the thief. Samurai, if I look 'em up in the Desktop 
Encyclopedia, are from Japan during the years 1000-1600 A.D. I'll write this down in my notepad 
for future reference. The 'More' button gives you clues as to the identity of the thief. Here (the 
more button) I get the clue that the thief had hazel eyes, and I'll write lhat down on my notepad 
as well. When I uncover data about the thief s identity I'll want to record them in my evidence 
folder. I look back in my notes and I see that one of the dues is that the thief is a male. I'll enter 
male under the sex option. I also know that the thief s eyes are hazel, so I'll click the eyes button 
until hazel appears. I won't compute yet to get an arrest warrant because I don't have enough 
information and it would just waste valuable Chronotime. so I'll continue my search. 

The next place I'll investigate is the informant. Here I learn the thief wanted to spy on the 
Anu tribe. Once again this clue will give me some idea as to the location the thief went to next. 
And if I look up the Anu tribe in the Desktop Encyclopedia, I'll find that the Anu tribe was part of 
Japan as well, so I can deduce from my Samurai clue and this clue that the next destination of the 
thief is Japan. When I click the 'Travel' button, I'll see that Japan is one of the destinations, and 
that's where I'll follow the thief to next. Notice that I did not go to the 'Scanner' for any clues. This 
would be a waste of Chronotime since the 'Scanner' only gives me clues as to the thief s next 
destination, and I was already able to deduce the thief s next destination based on the two dues I 
received previously. 

As in the previous location- I'll start my search by going to the Witness. O.K., a V.I.L.E. 
henchman has been detected. 1 his is feedback the game gives me to tell me I have correctly 
followed the thief to the next destination. If I hadn't followed the thief to the next destination, the 
witness would have said to me, he had never heard of that person, or doesn't know anything 
about what I'm talking about. The witness tells me that the thief wanted to go to Lima, so I write 
this down on my notepad, and I look up Lima in my Desktop Encyclopedia, and find out that 
Lima is the Capital of Peru. This gives me a pretty good idea that the next destination of the thief 
is Peru. 

I'll click the 'More' button to find out another clue as to the thief s identity. Here I find that the 
thief was holding a copy of The ^Hunchback of Notre Dame. This gives me some idea as to the 
author the thief liked. I'll immediately record this clue on my notepad as well as in my evidence 
folder. I'll click the 'Author' button and the first author that appears is Doysteyevski. I'll look up 
Doysteyevski in the Desktop Encyclopedia and see if he had anything to do with a book about the 
Hunchback of Notre Dame. He doesn't, so 111 have to go on to the next author. The next author is 
Victor Hugo. I look him up in the Desktop Encydopedia and sure enough he is the author of The 
Hunchback of Notre Dame. Since I have tiKree clues, this is usually enough to compute an arrest 
wairant, and sure enough the clues match a particular suspect. 

Now tliat I have an arrest warrant for the thief, I have to track him as quick as possible to the 
next destination. From the due the v\ntness gave me, I know his next destination is probably Peru. 
So I don t need any more clues, and I'll travel directly to that location. Once again, I'll begin by 
searching for clues...ahh, I've found another V.I.L.E. henchman this means I still must be on the 
right track of the thief. The clue the witness gives me is that the thief wanted to meet the founder 
of the Capetians. I write Capetians down in my notepad, and look it up in my Desktop 
Encyclopedia. The Capetians are from France, so I can be pretty sure that the next destination of 
the thief is France. Since i'm pretty sure of that clue, I won't waste valuable time searching for 
other clues; I'll go directly to the tliief s next destination. 
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I'll start once again by searching for clues at the witness. It looks like the thief is trying to kill 
me. This means he must be at one of the other sources of information at this destination. It also 
means that if I don't have my arrest warrant, I'd better go get it. Otherwise, I'll be charged with a 
false arrest. But since I have my arrest warrant, I can go searcliing for the thief to try and catch 
him. Let's try the scanner to see if that will locate the thief. There he is, now the capture robot will 
go do the work. Good, we've captured the thief. We'll now send the data to the crime computer to 
verify his identity, and we'll get a congratulatory message from the Chief. That's how you play 
the game Where in Time is Carmen San Diego. Remember to record notes of clues you uncover 
during your search, and to use the Desktop Encyclopedia to interpret those clues and determine 
the next destination or identity of the thief. Also remember to get an arrest warrant before 
capturing the thief, now be given a chance to play the game yourself. Good luck. 
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Appendix C - Transcription of Novice Movie 

"In this demonstration we'll be watching a first-time player play the game Where in Time is 
Carmen San Diego? My name is Peter and I'll be commenting on the player's performance as we 
watch the game. In this case an unknown male has stolen the Indian Constitution from India in 
1947. The player has proceeded to the scene of the crime, and has thirty-three hotu-s in which to 
solve the case. Let's watch as the player begins her investigation. 

Here we see the player searching for clues at the witness. This particular clue about a gang of 
Samurai gives the player a tip about where the thief went to next. However, the player does not 
write down the clue or does not look at the Desktop Encyclopedia in order to decipher the clue. 
While the player was at the witness, she missed clicking on the 'More' button. The 'More' button 
would have given her a clue as to the identity of the thief. This would have helped her to get cin 
arrest warrant. But now we see that she wants to go travel somewhere. 

Without deciphering the clues the player received at the vrftness, regarding the Samurai 
gang, she has travelled randornly to the United States. Let's see what happens now. As we can see 
the witness has nothing the tell the player. This me.^ns the player has travelled to the wrong 
location. What she needs to do now is get back to her previous location and get back on this trail 
of the thief. The player has now returned to India in the 1900's, therefore she is back on the trail of 
the thief. Notice the Chronotime is at 27 hours. Each time the player travels to a destination, or 
obtains a due from the witness, infoiTnant, or scanner, time is counted off the total chronotime. 
This means you have to be sure of where you're travelling, or to be efficient in searching for your 
clues so you do not run out of chronotime. Running out of chronotime means you vnll not be able 
to catch the thief. Let's watch now as she continues her search . 

Here the player gets the clue of the Shinto Shrine. She writes it down on her notepad and 
looks up 'Shinto' in her Desktop Encyclopedia. Once again the player forgets to click the 'More' 
button under the witness to get a clue as to the identity of the thief. 

In the informant, the player finds the due of the Golden Hall of Haruji. She writes these 
words down in her notepad, and looks up both Golden Hall and Haruji in the Desktop 
Encyclopedia. She finds nothing for these terms. Based on the clues she received from the witness 
and the informant, the player seems to have deduced that the next destination of the thief is 
Japan. What we have just witnessed is the appearance of a V.LL,E. henchman. These V.I.L.E. 
henchmen check up on you when you are following the thief correctly. Tlius, the player has 
correctly followed the thief to Japan. In the informant the player gets the clue of Austin & Emma. 
This clue gives the player some idea of where the thief travelled to next. She writes the clue down 
on her notepad, and looks up Austin in the Desk Encyclopedia. Once again the player missed 
clicking on the 'More' button to get a clue as to the identity of the thief. In the witness, the player 
gets the clue of Rugby. She writes Rugby down on her notepad and looks up Rugby in the Desk 
Encyclopedia, however, cannot find an entry for it. Once again, the player has not clicked on the 
'More' button in order to get a due as to the identity the thief. Without these types of clues, the 
player will not be able get an arrest warrant, thus, ihe player will not be able to win the game. It's 
very important that, when you do not have an arrest warrant, you always click the 'More' button 
to get some idea of who the thief is. 

It seems that from the dues of Austin & Emma and of Rugby the player was able to deduce 
that the next location that the thief travelled to was England. Let's see if that turns out to be 
correct, "^once again the V.I.L.E. henchman has appeared, so the player has successfully tracked 
the thief to the next destination. Here (under button which has just been pushed and illustrated 
on the screen) the player receives the clue of a mousetrap. She writes it down on her notepad and 
looks it up in the Desk Encyclopedia, but finds nothing. I tend to ^feel that the scanner is a waste 
of time, most of the dues that you need to ^follow the thief can be found in the witness and 
informant. Plus, the scanner will never have a due as to the identity of the thief. I usually only the 
use the scanner as a last resort if I cannot figure out the destination of the thief from the clues I 
obtain in the witness and informant. In the witness the player gets the due Titanic. She writes it 
down on her notepad, and looks it up in the Desk Encyclopedia and finds a reference for it. 
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Based on the clue of the Titanic, the player has deduced that the next destination of the thief 
is England in the 1900's. Note that back in the witness, she did not click the 'More' button once 
again. This denied her of an important clue as to the identity of the thief, which would have 
helped her get an arrest warrant. When a life- threatening trap has been set against the player, as 
the one that we just saw, this means that the thief is somewhere at this destination and that the 
player has to try to find the thief s whereabouts at either the witness, informant or scarmer. It 
appears that the player has finally realized that she needs an arrest warrant to catch the thief. 
Since the player did not investigate any of the 'More' buttons to obtain clues as to the identity of 
the thief, the only due she has is that the thief is a male. Usually- you need three or more dues in 
order to obtain an arrest warrant. So I do not think this player will be getting an airest warrant 
this time. As I mentioned, only one clue is insuffident data, and returns a whole list of possible 
suspects. Thus an arrest warrant cannot be issued. 

Another trap has been uncovered, the player is now getting dose to capturing the thief. The 
player has located the thief, and now the capture robot will be sent to make the arrest. Well, the 
player has successfully tracked Russ T. Hinge but unfortunately, without an arrest warrant, Russ 
T. Hinge is allowed to go free. Although the player made a minor mistake at the beginning of the 
game, she was able to successfully track the ti\ief from destination destination. However, she 
did forget to acquire clues as to the identity of the thief which prevented her from getting an 
arrest warrant. You'll now be able to have a chance to play the game Where in Time is Carmen 
San Diego. Good luck. 
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Appendix D - Data Collection Instrument 
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Subject: 
Treatment; 
Coach Name; 
Observer Name: 



Destinations 
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Sun Time; 
End Time: 



Task #2 
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Location/date 



Chronotime 



Witness 



Informant 



Scanner 



Note taking 



Reference 



Program doc. 



Select clues 



select lnf«nr«cl du<« 



Arrtst warrant 
attempt 



Arrest warrant? 



Task complete 



False arrest 



False leads 
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